Skip to content

Conversation

larsjfo
Copy link
Member

@larsjfo larsjfo commented Oct 14, 2026

Description

I have created a script to import, and update the dataset from Pokemon Showdown. It is broken down into three parts:

  1. Import the data from Showdown
  2. Transform the data into a format that is more convenient for our use case
  3. Load the data to MariaDB

A lot of the boilerplate of this import process has been aided by Claude Opus 5.5. Every single AI assisted line has been read and approved.

As a part of this PR I have also created schema.sql and some general database connection boilerplate. While I recognize that in an ideal world we would have done this in a separate PR, I felt the need to do it this way to make sure the code actually ran like I expected.

Unit tests are currently missing, might be added after Vitest is set up depending on whether we find it necessary.


Issue

@larsjfo larsjfo requested a review from a team as a code owner October 14, 2026 08:32
@larsjfo larsjfo self-assigned this Oct 14, 2026
@larsjfo larsjfo added this to the Andre underveislevering milestone Oct 14, 2026
@larsjfo
Copy link
Member Author

larsjfo commented Oct 14, 2026

@agnesas Denne er til deg :)

Copy link
Member

@agnesas agnesas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The importer runs successfully on the group VM, and npm run typecheck passes. I also verified that the database row counts match the import summary.

One thing worth investigating before merging: npm audit reports 11 dependency vulnerabilities, including one critical issue through transitive dependencies of pokemon-showdown. I would avoid npm audit fix --force, since it proposes a breaking downgrade. Could we check whether compatible updates are available?

There are also two small documentation issues: the incomplete “Setting up the” heading and the difference between the example database name (mydb) and the default (pokemon).

@larsjfo
Copy link
Member Author

larsjfo commented Oct 15, 2026

The packages with issues have been overridden individually. This should resolve the issues. I have manually tested that it works as expected.

The documentation uses mydb intentionally. This is to be more general, as it does not matter what the production name of the db is, it will just be whatever is in the .env file. However I did add en underscore (my_db) so it matches the style of the other example names. I also added the same example values to the example .env format in the documentation itself.

Copy link
Member

@agnesas agnesas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@agnesas agnesas merged commit 450f97a into main Oct 16, 2026
@larsjfo larsjfo deleted the 22-import-data branch October 18, 2026 10:24
@larsjfo larsjfo mentioned this pull request Oct 18, 2026
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prepare dataset for import Implement database schema Design database schema Create database and tables Import data
2 participants